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ABSTRACT 



As part of the "National 'What Works' Evaluation for Adult 
ESL Students, " a meeting of English-as-a-Second-Language (ESL) literacy 
experts and stakeholders was convened (May 10, 1996) to examine what theory 
and practice have to say about how ESL students acquire literacy skills, 
prevalent instructional approaches and methods, assessment issues to be 
addressed, and key variables in adult ESL learning. The report first 
summarizes five main issues raised during the meeting: (1) a need for 

guidance to inform policy and practice; (2) a need to recognize that adult 
basic education and ESL are different, and ESL learning processes pose more 
problems; (3) identification of the goals and purposes of ESL; (4) 
characteristics of ESL learners that may affect learning; and (5) how the ESL 
instructional environment affects learning. It then lists the key variables 
seen as affecting second language learning and literacy development and 
potential by-products of the study for assessment and instruction. A list of 
presenters and attendees at the meeting is included. (Adjunct ERIC 
Clearinghouse on Literacy Education) (MSE) 
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TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL 



NATIONAL "WHAT WORKS" EVALUATION FOR 
ADULT ESL STUDENTS 



Supported by the U.S. Department of Education (Office of Vocational and Adult 
Education and the Planning and Evaluation Service), the National “What Works" Evaluation for 
Adult ESL Students will identify and evaluate the effectiveness of instructional approaches and 
methods for adult ESL learners with limited literacy skills. The five-year evaluation, conducted 
by the American Institutes for Research and its subcontractor, Aguirre International, will address 
the following questions: 

■ What are the characteristics of adult ESL learners with limited literacy skills and 
how do learner characteristics impact program effectiveness? 

■ What program methods and instructional ^preaches are most effective in 
improving the English literacy skills of adult ESL learners with limited literacy 
skills? 

■ What fypes of assessment and outcome measures are used to evaluate the 
effectiveness of ESL programs for adult learners with limited literacy skills and 
which are most appropriate?. 

■ How do factors such as program administration, staffing, and per-pupil costs 
impact on program effectiveness? What approaches are most cost-effective? 

There are three components to the evaluation: 

■ Mail Survey of ESL Providers. A mail survey of ESL providers in the six states 
with the largest populations of adult ESL learners — California, Florida, Illinois, 
New Jersey, New York, and Texas — will provide general information about 
instructional methods, approaches, and assessment methods of programs. 

■ In-Depth Study of ESL Programs. About 80 ESL programs will be selected for 
more in-depth study through a telephone survey that will also collect information 
from approximately 250 local sites. Project staff will then select 40 sites to visit 
to conduct interviews with administrators and teachers, observe instruction, and 
hold focus groups with adult learners. 

■ Comparative Study at 16 Sites. The comparative study will include 16 sites that 
will follow a total sample of 3,200 adult ESL learners. An experimental or quasi- 
experimental design will be used to compare the effect of instructional methods or 
approaches on learning gains. Project staff will conduct semi-annual site visits to 
the study sites to interview administrators and teachers, hold focus group 
interviews with adult learners, and observe instruction. 
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Ms. Lenore Yaffee Garcia, Project Officer 
U.S. Department of Education 
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600 Independence Avenue, S.W. 
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e-mail: Lenore_Garcia@ed.gov 
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Introduction 

Research literature in the areas of English as a Second Language (ESL), literacy, and related 
fields can serve to inform our knowledge of effective practices for adult ESL students. To ensure 
that such information is accessed and represented in the National “What Works” Evaluation for 
Adult ESL Students, a group of leading experts in adult ESL literacy research and practice convened 
to lead a discussion with the project’s study team and other stakeholders (a list of participants is 
attached). At this meeting, held at the U.S. Department of Education on May 10, 1996, presenters 
examined what theory and practice have to say about how ESL students acquire literacy skills, as 
well as what prevalent instructional approaches and methods are used, what assessment issues need 
to be addressed, and what key variables are likely to have an impact on adult ESL learning in 
general. 

This report summarizes issues and ideas raised in the meeting, beginning with issues 
affecting the adult ESL field (such as differences between Adult Basic Education (ABE) and ESL, 
and characteristics of adult ESL learners). Also discussed were issues in the area of language 
acquisition and literacy development, including how adults learn, instructional approaches, and the 
role of native language in adult education. 

Field Issues 

Participants began by noting that there is little consensus in the field on the characteristics of 
effective instruction or on the outcomes that are appropriate for evaluating adult ESL. For example. 
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only California has developed adult ESL instructional standards. However, five main issues were 
seen by meeting participants to confront the field: 

■ the need for guidance to inform policy and practice, 

■ the need to recognize that ABE and ESL are different — ESL is not just “ABE with an 
accent,” 

■ goals and purposes of ESL, 

■ characteristics of ESL learners that may affect learning, and 

■ how the ESL instructional environment affects learning. 

A summary of the discussion of these issues is presented below. 

Guidance needed for policy and practice 

The field would find it particularly useful to have research-based evidence to learn exactly 
what students need to learn and how, as well as what content and competencies need to be taught. 
Research is difficult, however, since the field does not currently agree on what constitutes best 
practices of effective strategies. In addition, given the diverse characteristics and needs of ESL 
students, there is probably no simple “best method” for everyone. Areas of special interest for 
research include: 

■ whether biliteracy is effective for improving English literacy: 

■ whether CBOs are better places for learners with limited literacy skills than are K- 12 
schools, due to availability of bilingual staff and the generally smaller size of CBOs, 
which may be conducive to students supporting one another; 

■ what qualifications instructors should have; 

■ whether instruction should be one-on-one or in groups; and 

■ how grammar and phonics ought to be integrated into the ESL literacy curriculum. 

Other issues include whether computer-assisted instruction will help and whether there is a “literacy 
threshold” that learners must attain before they can benefit from learning. It was noted that when 
examining instructional approaches, implementation should be considered along a continuum, since 
instructors typically use a variety of approaches. Consequently, researchers need to examine the 
extent to which approaches are implemented and avoid an “either-or” orientation. 
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Although the field is looking for guidance in the areas of policy and practice, it should be 
noted that some of those in the field tend to be somewhat skeptical of large-scale designs, essentially 
because such designs, which typically employ closed-ended questions, disallow the opportunity to 
tell an individual program’s “story;” and the data that result from these studies are sometimes not 
seen as useful. 

Differences between ABE and adult ESL 

The greatest difference between ABE and adult ESL is that adults who are ESL literacy 
students do not yet know English and need to acquire a reading and writing system along with 
English proficiency. For this reason, the process of learning is more difficult for them than it is for 
ABE students. Furthermore, ESL students must contend with acculturation issues. With regard to 
instructors, many in the field have noted that, as a group, ESL teachers “seem more committed and 
sophisticated” than ABE instructors. While often there is a stigma attached to attending ABE 
classes, ESL, on the other hand, has no stigma attached, since it is seen as a developmental and not a 
remedial process. In ESL, group learning is more central to instruction, since the acquisition of 
language is seen, in part, as a social process. ESL and language acquisition research, in fact, support 
such group learning, as social interaction supports the practice of newly acquired English skills. 
Finally, goals for ESL students tend to be broad (they want to “learn English” and be able to use it in 
a wide range of contexts). Conversely, ABE students are more focused in their individual goals. For 
example, many are in ABE classes to earn a GED or to attain other specific goals, such as a driver’s 
license. 




Goals and purposes of ESL 

Some of the goals of adult ESL instruction are to help learners to: 



■ become literate in English, 

■ acquire communication skills, 

■ prepare to enter other learning institutions, 

■ negotiate “the system,” 
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■ get a foundation in the structural components of the second language, and 

■ develop strategies that help them to get by without knowing much English (strategic 
competence). 

Characteristics of adult ESL literacy learners 

Most adult ESL “literacy” learners have not earned a high school diploma, and in many cases 
have had only three to five years of schooling. Common language backgrounds include Spanish (the 
majority), Hmong, Cambodian, Vietnamese, and Haitian Creole; and the population consists largely 
of young men who are “on the go,” as well as older women whose main goal is to both help their 
children and function in a social atmosphere. There also are significant groups of people who live in 
linguistically isolated communities, such as El Paso (Spanish) and Fresno (Hmong), which limits the 
learner’s opportunity to use English outside of the classroom. Furthermore, in some classrooms, 
there are students who know some English (i.e., oral — aural) and are able to communicate face to 
face, and there are those who know no English and have to learn both oral English and reading and 
writing. These factors may determine how well a learner is able to acquire a second language and 
whether different instructional strategies are needed. 

Learning environment 

Adult ESL classes take place in a variety of locations, including ABE classes, CBOs, 
libraries, technology-based labs and centers, work places, union halls, and prisons. Computers may 
or may not be used, and class sizes may consist of small groups, large groups, or individual students. 
These various settings and configurations also may affect learners’ acquisition of a second language, 
as well as the instructional strategies, which may differ greatly. 

Research on Language Acquisition and Literacy Development 

Existing research addresses a number of issues in the area of language acquisition and 
literacy development including how adults learn, what instructional approaches are used, the role of 
native language, and outcomes and assessments. From a discussion about these issues, participants 



proposed a list of key variables most likely to have an impact on adult ESL learning. That discussion 
and a list of variables are summarized below. 

How adults learn 

Adult learning theory holds that adult learning is best accomplished by linking the 
instructional approach to the learner’s experiences and goals. In terms of ESL, language acquisition 
research indicates that language learning requires comprehensible input, that is, learners have access 
to language they can understand (Krashen’s theory). The level of instruction and the learner’s level 
of language proficiency should be linked, because individuals learn best at one level above where 
they actually “are.” This implies that the “sink or swim” approach of mixing ESL learners with 
English-speaking ABE students is not effective. 

Second language acquisition requires attention to both form and function, and teaching 
should include the following components: 

■ fluency (talking and getting the point across, as well as understanding “natural” speech), 

■ accuracy (saying and writing the language correctly), and 

■ social appropriateness (using the language in ways that fit the situation). 

Furthermore, hypothesis testing, in which a learner is able to discover how a language “works,” is an 
important aspect of language acquisition. 

Students do not learn a second language by memorizing structures and then using them 
correctly. Rather, they form a hypothesis of how language works and then come to understand how 
language works, through a cognitive process that includes predicting meaning, monitoring, and 
confirming (or changing) original hypotheses. 

Instructional approaches 

As the term is used in the field, an instructional approach links a theory of language 
acquisition with a theory of learning, and includes a philosophy of teaching and learning as well as 



ERIC 



5 



8 



instructional principles. An approach, such as the Whole Language Approach, also weaves teaching, 
learning, and assessment into a coherent whole. 

Since any approach, in theory, is meant to be self-contained (addressing all aspects of 
teaching, learning, and testing), it does not make sense to combine approaches. In practice, however, 
teachers tend to implement certain aspects of an approach (but not others), or supplement an 
approach with activities reflecting methods that may appear to be contradictory. 

While an approach in theory tends to be fairly broad, the concept of instructional method is 
more limited. Methods usually have some theoretical foundation (e.g., behaviorism forms the basis 
for the audio-lingual method), but do not try to account for all aspects of language and literacy 
learning. A method also tends to be much less concerned with beliefs and principles than an 
approach, and generally outlines a set of teaching practices meant to promote language acquisition 
and literacy development. The Total Physical Response Method, for example, focuses on helping 
learners to understand the structure of the language by asking them to listen to various commands 
and to respond nonverbally to those commands. The method does not “guide” teachers who want to 
help learners to develop a strong background in literacy. 

Although the literature tries to distinguish between “approaches” and “methods,” 
practitioners often use the terms interchangeably. The issue is further complicated by the fact that 
the Language Experience Approach technically is considered a “method,” since it does not provide 
principles that might guide teachers in their efforts to build oral communication skills (or more 
advanced literacy skills). 

Instructional techniques constitute individual teaching strategies or instructional practices 
that can be combined in various ways — and may reflect a certain approach or method, although the 
teacher may not be aware of the assumptions that form the basis of certain techniques. In practice, 
the techniques used often are part of a teacher’s “bag of tricks” (e.g., what always works), or are 
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driven by the activities in a textbook. Having students generate stories about their lives or everyday 
activities is an example of a technique used with the Language Experience Approach. 

Research indicates that approaches with a “basic skills” orientation, though widely used, are 
not entirely effective in promoting full language proficiency. Despite the fact that many teachers and 
learners believe that “real” language learning is learning basic skills such as grammar, this approach 
is generally found to be ineffective. However, language learning for adults does require some 
attention to both form and function. Therefore, it is possible, and, in fact, useful, to integrate such 
“basic skills” approaches into the whole language approach. When doing so, however, it is 
important to use words with which the learners are familiar, or words that are already part of the 
lesson. “Breaking down the language” is the means toward a larger goal. Learners need 
competencies they can use for their own purposes. 

Role of native languaee in second language acquisition 

Adult Second Language classrooms are becoming increasingly segregated by language, and, 
even in multilingual adult centers, those learners with “language-similar backgrounds” tend to 
cluster. The use of native language (even if the teacher does not know the language) might, 
therefore, be an underexplored resource, in that it may be useful for learners to resort to their native 
languages with peers, in some situations. In fact, according to Cummins, it may be necessary to 
learn literacy only once. The acquisition of literacy in the native language, then, may be a 
particularly key element of learning English for ESL students with limited literacy skills. 

Cummins argues that there is a certain threshold for literacy in the native language that 
learners should reach before they move on to acquiring a second language. Below this point, it may 
be impossible to acquire the cognitive academic skills that are part of language. Cummins’ work has 
been with children, however, and has not been extended to adults. Testing this hypothesis and 
finding this threshold in adults would be significant, but is beyond the resources of the What Works 
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study. However, a Center for Applied Linguistics (CAL) study found that children who had literacy 
in any language fared better in acquiring another language than those who had no literacy. It could 
be a major contribution to the field, if research demonstrates the relationship between native 
language literacy and English acquisition in adults. 

Another issue that arises in this area centers upon what a learner’s native language might be. 
Learners who move to the United States in their younger years and come of age here may not have a 
good grasp of either their “home” language or English. When these children become adults, they 
may enter adult literacy classes wherein their literacy needs in both languages must be addressed 
before progress can be made. 

Determining a learner’s initial literacy level in a native language also is problematic. 
Currently, at the program level, determination of a student’s literacy level is accomplished at first 
intake. Assessment procedures are often informal. For example, students fill out forms either in 
English or in their native language, and programs determine the individual’s literacy level by how 
well the student completes the form. Alternatively, programs may ask learners about the years of 
schooling they have completed, and use those data as proxies for the determination of a literacy 
level. The What Works study may use these methods; it also may be possible to apply self-report 
methods used by NALS (but the accuracy of this measure is uncertain). A forthcoming NALS report 
may shed light on this topic, as it contains an analysis of self-report with performance. 

Some ESL programs teach biliteracy — literacy in both English and the native language. 
Such programs vary a great deal, ranging from maintenance programs, which are intended to build 
up native language literacy skills of learners with limited literacy skills (in adult education 
maintenance programs are quite rare), to transitional programs, which move learners toward the use 
of English only. Bilingual programs attract students, perhaps because of the promise of 
communicating immediately that such programs convey. Furthermore, such programs use a learner’s 



own strength in his or her native language to develop literacy, and, ultimately, English skills. Some 
researchers believe the success of biliteracy programs is partly because learners who are proud of 
their first language and culture find it easier to acquire a second language. On the other hand, other 
practitioners believe that learners who are most anxious to acculturate may resist learning literacy in 
their native language and may not learn well in biliteracy programs. 

Despite their popularity, enrollment in K-12 bilingual programs has declined in California, 
partly due to anti-immigrant sentiments. Furthermore, children may acquire a negative attitude 
toward the home language, due to anti-immigrant messages they receive in the classroom (or from 
their own parents’ experience of learning English). Participation in bilingual programs also concerns 
some families, because such participation may indicate that someone is here illegally. It is important 
to take such contexts into account when planning ESL programs, because they can have an impact 
upon what occurs in the ESL classroom, as well as on funding decisions. For example, a learner may 
be discouraged from attending bilingual classes because it appears that English is not being taught. 
Distance learning may then become more attractive to learners (an examination of the Crossroads 
Cafe project may be useful in order to understand this attraction). 

Outcomes and assessments 

Some confusion exists in the field regarding what is meant by “what works,” due to a 
mismatch between outcomes and assessment measures. For example, students may make a great 
deal of progress in reading to their children, but, if they are tested on a standardized test, such as the 
TABE, this progress is not reflected. Therefore, it is important to link any assessment to what the 
program teaches, in order to connect to the major focus. The use of standardized tests cannot do this, 
unless the instructor specifically “teaches to” the standardized test (and the test has content validity). 

States and programs are now being held responsible for establishing “new” outcomes, such 
as whether or not learners are writing better, whether they transition to other programs, become more 
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involved as citizens, or obtain jobs. Current assessments do not measure such outcomes. Portfolio 
assessments have been used for the assessment of writing, but there are no standards in adult 
education that can help teachers to evaluate writing samples; students track their own progress, to an 
extent, but improvements are not documented in a systematic way. 

The issue is not so much that outcomes do not exist, but that the ESL field has varying 
objectives for instruction and does not know how to measure outcomes across groups. ESL 
instruction builds literacy, if learners continue to receive instruction for several years. However, for 
students who remain in programs only for a short time, a determination needs to be made as to what 
students can and should accomplish in that time period — and how accomplishments should be 
measured. Teachers need to think about what it is they want students to learn by the end of the class. 
Part of teacher training may, therefore, need to highlight the process of assessing and monitoring 
student progress in the classroom, on a daily basis. Determining accomplishments and monitoring 
progress is not only the instructor’s responsibility, though, as learners and programs need also to be 
involved. 

Key variables that impact adult ESL learning 

Participants listed the following key variables they thought made a difference in second 
language acquisition and literacy development. 

■ philosophical coherence of instruction, assessment and teacher training, 

■ involvement of teachers, program staff, and learners in program planning, developing, 
and implementing the program, 

■ use of L] (native language) in instruction, 

■ use of assessment methodology and outcomes that reflect what is taught, 

■ opportunities for learners to use English both in and out of class, and to use both English 
and the native language for literacy purposes, 

■ teachers qualities and teaching environment, including such factors as training in areas 
related to instruction and in identifying learner goals, teacher experience with ESL 
learners with limited literacy skills, teacher salaries and resources available to teachers, 
and 
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■ use of technology, the extent to which it is used to support literacy and language 

acquisition, and the extent to which availability indicates “dollar support” for a program. 

Other variables of interest to examine in the evaluation that were mentioned by participants 



include: 



■ the extent to which learners have opportunities for social interaction, 

a program focus (e.g., skills-based, whole language approach), 

a whether classes consist of a single language group or a mixed language group, 

a degree of individualization, including integration of learner goals and interests into 
instruction, and whether or not there are different groups of learners for whom different 
instructional strategies are used, 

a time (e.g., duration and intensity of institution), 

a ties to other programs in the community — so learners know where they can go next, 
CBOs, businesses, and for support and transitions, 

a voluntary versus mandatory enrollment, 

a student access to support services (e.g., child support, transportation, bilingual and 
culturally competent counseling, health-related services), 

a extent to which programs facilitate and promote language learning outside the classroom, 
and 

a use of tutors. 

Participants noted that the variables should be thought of as continua and in the study, we should 
consider selecting sites at the ends of each continuum. In addition to identifying effective 
instructional practices for adult ESL literacy students, participants noted that the study could produce 
several bi-products that would be of great assistance to the field. Potential by-products could 
include: 



a Measures and procedures for assessing learning gains and outcomes of adult ESL literacy 
students; 

a A program self-assessment instrument that could be used for self-evaluation; 

a Identification of the role native language literacy plays in second language learning; 

a Identification of innovative instructional practices, such as use of the Internet, sending 
e-mail, or other uses of technology. 

The issues and ideas raised in this discussion have important implications for this study, in 
terms of both refining the survey instruments and, ultimately, interpreting results. Ensuring this 
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information is represented in all aspects of the study will help us to meet the study’s goal: to provide 
guidance to the field on what works well, and to demonstrate to policy makers which practices are 
effective and worth supporting. 
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